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ABSTRACT 

The Educational Testing Sen’ice (ETS) created the Graduate Major Field Test in Business (GMFT-B) 
for MBA students. This test is administered to all MBA classes at Jacksonville University for the 
purpose of measuring student academic achievement and growth, as well as to assess educational 
outcomes. The test is given in the capstone course, Business Strategy and Policy, typically taken by 
students at the end of their MBA program. It provides valuable feedback to each student as to their 
standing with regard to students across the nation. A total of six years of data (2006 to 2011) was 
collected for analysis of these test scores. For the first half of this time period, Jacksonville University 
was in candidacy for accreditation by the Association for the Advancement of Colleges and Schools of 
Business (AACSB), and for the time period 2009-2011 it was an AACSB accredited institution. 
Executive MBA, evening MBA (also known as flex MBA) and day-time (accelerated) MBA classes 
took the exam over that time period. It was determined that the executive MBA student grades were 
significantly better than their counterjrarts in the evening (flex) and day-time (accelerated) MBA 
students at Jacksonville University. It was also found that there was no significant difference in 
student performance during the first half (AACSB candidacy) and second half (AACSB accreditation) 
of the time period under investigation. 
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INTRODUCTION 



he Educational Testing Service (ETS) in Princeton New Jersey administers a number of major field tests 
in a variety of academic disciplines. These are at the undergraduate and graduate areas and cover some 
of the following subjects: Biology, Business Administration, Computer Science, Economics, and 
Psychology. The purpose of the tests is to measure student academic achievement and growth and to assess the 
educational outcomes of various major programs. Development of the Major Field Tests (MFTs) began in 1989 and was 
modeled on the development of the Graduate Record Examinations (GRE) subject tests. Unlike the GRE, the MFTs do 
not serve as a predictor of graduate school success, but are designed to measure the basic knowledge and understanding 
achieved by MBA students (Educational testing Service, 2006). 


Experienced faculty members representing all of the relevant areas of a discipline determine the test 
specifications, questions and types of scores reported. The test assesses mastery of concepts, principles and knowledge 
by graduating Business students. In addition to factual knowledge, the test evaluates student’s ability to analyze and 
solve problems, understand relationships and interpret material. ETS assessment experts subject each question to 
rigorous tests of validity and reliability. To ensure fairness and content relevance, the test is revised approximately every 
five years. A Comparative Data Guide is published each year containing tables of scaled scores and percentiles for 
individual student scores, departmental mean scores and subscores or group assessment indicators that the test supports. 
Nearly 1500 colleges and universities employ one or more of the MFTs each year (Educational Testing Service Major 
Field Tests, 2011). According to ETS, the Major Field Test in Business (MFT-B) provides institutions with 
information to use in curriculum evaluation, departmental self-study, and end-of-major outcomes assessment (ETS 
Handbook 2007). For example, Ursuline College used the MFT-B in such areas as psychology to identify program 
and curriculum changes that might enhance students' performances and help them to become better prepared 
applicants in the graduate school process (Frazier and Edmonds 2002). 
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ETS MAJOR FIELD TESTS IN BUSINESS 

Business schools are increasingly being required to locate sources of direct measurements that confirm and 
ensure student learning. Pressures are being exerted by such accreditation bodies as the Association for the 
Advancement of Collegiate Schools of Business (AACSB) International, regional university accrediting organizations 
such as the Southern Association of Colleges and Schools (SACS),and stakeholders such as lawmakers, students, 
alumni, faculty, administration, and community leaders. These stakeholders are increasingly demanding 
confirmation that business colleges demonstrate value to their students by producing knowledgeable and skilled 
graduates. Colleges and universities are now required to show that they are gathering data to demonstrate that their 
programs are being successful in this mission. This is often referred to in AACSB parlance as “closing the loop” 
(Wilson, 2008). 

AACSB implemented new accreditation standards in 2003 that required business schools to assess student 
learning using new methods. An excellent source of direct information that confirms discipline specific learning that 
was readily available was the MFT-B. While the MFT-B is an excellent instrument, administrators and faculty are 
sometimes mystified how to review and interpret the test results. Little has been written about the success (or 
failure) arising from the use of standardized tests for outcomes assessment, and there is no prescribed process for 
moving from viewing results to improving academic programs and few examples were in the literature 
(Mirchandani, Lynch, and Hamilton 2001). However, over one-third of AACSB schools surveyed shortly thereafter 
reported that they were using the MFT to assess programs (Pringle & Michel, 2007). 

The MFT-B is specifically designed to assess mastery of concepts and principles as well as knowledge 
expected of students at the conclusion of a major in specific subject areas (ETS On-line Webinar 2007). Knowledge 
areas include Accounting, Economics, Management, Quantitative Business Analysis, Finance, Marketing, Legal and 
Social Environment, Information Systems, and International Issues. MFT-B questions are constructed and reviewed 
by committees of experts in each subject area. The test goes beyond measurement of factual knowledge. It also 
evaluates a students' ability to analyze and solve problems, understand relationships, and interpret material (ETS 
Handbook 2007). The MFT-B provides college and departmental administrators with important quantitative direct 
measurements of student learning to assess the effectiveness of their programs. Standardized test scores provide 
external validity and allow an institution to establish a baseline and benchmark against national norms 
(Mirchandani, Lynch, and Hamilton 2001). However, there are some cautions. Bycio and Allen (2007) recommend 
that great care be taken in interpreting the MFT-B scores. The meaning of the score is difficult to interpret without 
knowing such things as the extent of student motivation taking the test. The information is provided via a number of 
tables that at times can be difficult to assess objectively. ETS importantly notes that the data should be considered 
comparative rather than normative. The institutions employing the MFT-B do not proportionally represent the 
various types of higher education institutions (ETS Handbook 2007). Nonetheless the MFT-B provides one of the 
few tools available to compare student learning across the nation. 

ETS GRADUATE MAJOR FIELD TESTS IN BUSINESS 

The Graduate Management Field Test in Business Administration (GMFT-B) was developed to provide the 
same information as the MFT-B at the graduate (MBA) level. This test has the same basic format as the MFT-B and is 
administered in a similar fashion. It contains one hundred and twenty multiple choice questions. This test is completed 
over a three hour period (two sittings of sixty questions over an hour and a half each). The subject areas are as follows: 
Marketing, Management (includes Organizational Behavior, Human Resources Management, Strategic Management and 
Operations Planning/Management Science), Finance and Managerial Accounting. Each major area is weighted roughly 
25% of the entire exam (ETS, 2005). 

The GMFT-B was administered at Jacksonville University starting in 2006. The test is taken in the capstone 
course. Business Strategy and Policy, taken at the end of the MBA program. The test provides valuable feedback to each 
student as to their standing with regard to students across the nation. A total of six years of data has been collected for 
analysis over the period 2006-2011. Executive MBA, Evening MBA and a day-time (Accelerated) MBA classes took the 
exam over that time period. 
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The GMFT-B has proved to be very useful at Jacksonville University. As the scores go from a maximum of 
300 to a low of 220, MBA students have been able to determine how well they performed vis-a-vis their peers at other 
institutions around the country. In addition, all students are provided valuable feedback on how well they do in each of 
the major areas such as finance, accounting, management, and marketing. The national average for the test is 248.9. A 
student scoring a 270 knows that they are more than one standard deviation above the mean and are in the top 15% in the 
nation. The high grade that has been achieved by one of the JU Executive MBA students has been a 292 which put that 
student in the top 99.5% of the nation. Conversely students scoring in the 235 range are in the bottom 20% of the nation. 

In addition to providing useful individual feedback, the GMFT-B provides valuable cumulative statistics for the 
DCOB. This data was provided to the AACSB accreditation team that visited in the fall of 2009. The DCOB was able to 
demonstrate that the mean scores of each class were above the national average and that several of its scores were one 
standard deviation above the national average. However, with data available from 2006 until 2011 it is felt that more 
useful information could be gleaned from this information. 

PROBLEM STATEMENTS 

With this in mind two major questions presented themselves for analysis. The first was whether or not the 
different types of MBA classes were performing differently on the exam. The formats of the MBA that are offered by 
the DCOB include the Executive MBA, Evening MBA and Accelerated (day time) MBA. As the accelerated MBA 
students are recruited in a very similar fashion to the evening students and take the same curriculum, it was felt that there 
were essentially two different formats, the Executive MBA and the traditional MBA. Thus the first problem statement 
was whether either one of these two formats were producing superior results. Secondly, there were major changes 
initiated in the DCOB in the 2008-2009 time periods to prepare for the AACSB initial accreditation visit in the late fall 
of 2009. Thus, the data breaks nicely into two pieces with the 2006-2008 data being pre-AACSB and the 2009-2011 
data being the post-AACSB period. Could this data show a change due to the major structural changes in the DCOB 
driven by the desire to achieve AACSB status? Thus, the second problem statement concerned whether or not the 
changes in the DCOB made to achieve AACSB status had been reflected in improvement in student scores. 

HYPOTHESIS 

Null Hypothesis #1: There is no relationship between the type of MBA (executive and traditional) and the score on 
the GMFT-B. 

Alternate Hypothesis #2: There is a relationship between the type of MBA (executive and traditional) and the 
score on the GMFT-B. 

Null Hypothesis #2: There is no relationship between the time period (pre-AACSB or post-AACSB) and the score 
on the GMFT-B. 

Alternate Hypothesis #2: There is a relationship between the time period (pre-AACSB or post-AACSB) and the 
score on the GMFT-B. 

These hypotheses were tested at the five percent level of significance. 


DATA 


The GMFT-B is administered for assessment purposes in the strategic management class of both the 
traditional and executive MBA programs. This sample was selected from the period of time starting in the fall 
semester of 2006 and ending with the summer semester of 2011. The only semester during this time period when 
the test was not administered was the summer of 2010. During this time span, 435 MBA students took the test. Of 
these 435 students, 366 are traditional MBA and 69 are executive MBA. The number of scores from the pre- 
AACSB time period is 189, while the number of scores from the post-AACSB time frame is 245. The overall 
results of the ETS graduate business test are summarized in table 1, as well as the results by type of MBA and time 
period. 
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Table 1: Descriptive Statistics for ETS Graduate Business Exam 


Group 

Sample Size 

Mean 

Std. Dev. 

Maximum 

Minimum 

All MBA 

435 

255.7 

14.0 

292 

220 

Traditional Only 

366 

254.4 

13.7 

288 

220 

Executive Only 

69 

262.5 

13.7 

292 

235 

Pre-AACSB 

189 

255.5 

14.5 

292 

220 

Post-AACSB 

246 

255.9 

13.6 

288 

225 


METHODOLOGY 

A multiple regression model was constructed to examine the impact of type of MBA program (executive or 
traditional) and time period (pre-AACSB or post-AACSB) on GMFT-B scores. The dependent variable of the 
model is GMFT-B scores. The first independent variable is a binary variable for type of MBA program. This 
variable is set equal to one if the student who took the test was a member of the executive MBA program. It is set 
equal to zero if the student was enrolled in the traditional MBA program. The second independent variable is also a 
binary variable. It is set equal to zero for tests taken during the pre-AACSB time period, and one for those taken in 
the post-AACSB time period. 

RESULTS 

Table 2 shows the multiple regression results. While the model has significant explanatory power (F = 
10.04 with an associated p-value of 0.000), it only explains 4% of the variation in GMFT-B scores. The null 
hypothesis that GMFT-B scores are not related to the type of program is strongly rejected. The sample evidence 
indicates that students enrolled in the executive MBA program can be expected to score 8.06 points higher than 
students enrolled in the traditional program. The null hypothesis that GMFT-B scores are not related to the time 
period when the exam was administered cannot be rejected. The p-value that corresponds to the test statistic for the 
estimated coefficient for this independent variable is much greater than the stated level of significance of five 
percent. The sample data suggests that there is no difference in GMFT-B scores in the pre-AACSB and post- 
AACSB time periods. 


Table 2: Multiple Regression Results 


Variable 

Estimated Coefficient 

Test Statistic 

P-Value 

Constant 

254.40 

t = 248.1 

0.000 

Type of Program 

8.06 

t = 4.5 

0.000 

Time Period 

0.03 

t = 0.02 

0.983 

Adjusted R 2 = 4.0% 

| F = 10.04 

| P-Value = 0.000 


The differences in the distribution of exam scores by type of MBA and time period are also illustrated by 
the boxplots in Figures land 2. 

CONCLUSIONS 

The significant difference in the performance of EMBA students and normal MBA students is clearly 
evident in the analysis. This begs the question “Why do EMBAs perform better than traditional MBA students”. 
Both receive the same MBA graduate degree and both take similar courses. There would appear to be three major 
differences in the two programs. The first is in the selection process. EMBA students are hand-picked by their 
respective employer organizations. As the cost is higher ($50,000 vice approximately $25,000), companies are very 
discriminating and send only their best, high achievers and high potential managers and executives. Secondly, the 
program functions differently. The EMBA is a lock-step eighteen month program where students meet every two 
weeks on Fridays and Saturdays. The courses are ordered in an optimal sequence with the necessary reviews in 
statistics, accounting, finance and economics built into the beginning of the program. The traditional MBA is done 
in a very flexible fashion. Students take the core requirements in the order that they prefer and may take up to five 
years to complete the program. The lapse in time between materials presented in the front of the program until it is 
tested by the MFT- B could be as much as five years (or more if the student gets an exception to the five year 
requirement). 
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Figure 1: Boxplot for ETS Scores by MBA Type 
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Figure 2: Boxplot for ETS Scores by Time Period 
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The last difference is in the cohort nature of the EMBA program. Students are placed in a four or five 
person group that is carefully constructed to contain as many of the necessary business skills as is possible i.e. 
operations, marketing, information technology, finance, accounting etc. It is not uncommon to find a CFO or a CEO 
in the various groups. Thus, there is much business knowledge self-contained in each of the cohorts. EMBA 
students often comment in their out-processing briefs that they learned a great deal of business knowledge from the 
actual members in each of the cohorts. 

The second major part of this analysis pertains to the performance of students in the pre- and post-AACSB 
classes. The JU Business College (Davis College of Business) received its accreditation in early 2010. Most of the 
processes were put in place in the 2008-2009 time-frames. Thus, by treating the scores in the 2006-2008 timeframe 
as pre-AACSB and the 2009-2011 scores as post-AACSB it was possible to evaluate if there was any difference 
between the two. It was concluded that there was no discernible difference between the two sets of scores. Thus, 
AACSB accreditation did not appear to have any significant effect on the performance of the various MBA classes 
at Jacksonville University. 

The results discussed above are important to the JU Davis College of Business. For instance, employers 
who are discussing which MBA program to send their students to at JU often ask why they should spend twice as 
much money for the EMBA program. The results above allow a rational and empirical basis for advising why the 
EMBA program provides superior results. 

RECOMMENDATIONS 

Since the Accelerated MBA program was launched in 2011, it has provided very little data in which to 
study how effective the program is compared with the other two MBA options. A follow-on study needs to be 
conducted when sufficient data is available from the Accelerated MBA classes. 

Another area that can be explored in a follow-on study is the variation in the scores in the individual subject 
areas that make up the GMFT-B test. The test provides results in each of the major areas of business, i.e. 
accounting, finance, marketing, management etc. This study only considered the overall score on the GMFT-B. It 
would be very interesting to see if the various major areas each contribute equally to the difference in scores 
between the EMBA and the MBA. 

Lastly, it would be useful to compare the results between the graduate ETS scores and the undergraduate 
ETS scores. Although the tests are different, each ETS test has the same goal, i.e. to assess the level of performance 
of the students in essentially the same business areas. Where discrepancies exist between the two programs could 
shed light on potentially effective teaching and delivery styles of pedagogy. 
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